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INTRODUCTION s> 



In the OECD paper *'A Framework for Educational Indicators to Guide Government Decisions" 
(OECD, 1973)1/, it is suggested that the International Association for the Evaluation of Educational 
Achievement (lEA) study produces indicators of knowledge and skills transmitted in the achievement 
scores in a series of subject areas in twenty-three countries. This is true in that the cognitive • 
oMtcomes of schooling as measured'by lEA cov.er not only the means and ranges of knowledge, but 
also of skills and attitudes. However, lEA hjis also attempted, through data collection and analysis 
in a series of countries, to identify those factors (indicators) which are inputs to schools and educa- 
tional processes in schools which in some senses influence, or at least^are associated with, the 
various levels of the output of the schools. Indeed, lEA'is major thrust has been to idientify the 
factors associated with variation between students and also between schools within a population 
within a country. 

This paper presents some of the main findings of the most recent lEA study (The Six-Subject 
Study) (2-7). First of all, let us recall the data collection of the lEA study. Table 1 presents the 
different subjects taken at the various population levels in each of the countries participating in the 
study. 

The definitions of the three populations are as follows: 

Population I - all students in full-tim^ schooUng aged 10:00 - 10:11 at 
the tine; of testing. 

Population II - all students in full-time schooling aged 14:00 - 14:11. 

Population IV - all students in the terminal year in full-time secondary 
education progranrtmes which. were either pre-university 
programmes or programmes of the seLme length. (The 
interpretation of this definition of this group varied as 
. ♦ well as tlfe percentage of an age-group in this target 

IMjpulation. ) ^ 

The above definitions held true for Science, Reading Comprehension, Literature and Civic 

Education. In the case of English and French as foreign languages, a further condition was that 

the studeijts should be currently studying the language and have studied it for at least two years. 

Indeed, when examining the home background distributions of the students taking English and 

French,' it was clear that where English was part of the curriculum it was studied by more or les^ - 

all of the students in full-time schooling, whpreas French tended to be restricted to smaller 

^segments of this population. 



1^/ OECD, A framework for educational indicators to guide government 
. decisions, OECD, Paris, April 19T3. 
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Table 1, Subject areas and populations tested by country 
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Reading Com- 
prehension 
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IV 


I II 
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II 
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X 






X 
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Belgium (Flemish) 
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X 
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X X 
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X 






X 
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X 




X 


X X 
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X 




X 






X 




England 
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X 
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X X 
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X 
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X X 
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X X 
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X X 
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X X 


X 
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X X 






X 


X 


X 


XXX 


France 










X 
























Hungax^y 




X 


X 




X 


X X 
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X X 
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X X 




X 
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X 


X 
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X 
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X X 
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X 
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X X 
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X X 




X ,x 


X 


X 


X 


X X 


Thailand ^ - 




X 
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X 
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X 




S, A. 
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X X 
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X 


X X 


X 








X . X 



a) Population III was not an internationally defined popula^tion. Each pax^icipation research centre was \i 
able to define a population somewhere between Population II and IV to test with Population II for 
national purposes. The analyses were national and not international, 

/ > * - * 
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Probability samples of schools and students within schools were drawn for each level for 
each subject (or group of subjects) within each country. The way in which the complex samples 
were drawn, together with the resultant standard errors of sampling and Design Effects (DEFFS) 
for selected variables have been shown in detail elsewhere. 1^/ 

Three to five years of development work were required to construct the outcome (i. e. p.er- 
formance) measures in each subject area at each level. A list of the major outcomes classified by 
cognitive and affective outcome is given in Aprsendix I. The resultant tests had good reliabilities 
and were judged by the subject-matter panels in the various countries to be apipropriate for testing 
what was, in general, meant to liave been learned in school up to that point of testing. The test 
formats used included multiple choice, open-ended and file-in formats. For foreign languages 
tape recorders were used fpr Listening Comprehension and Speaking. Both mark-sense cards and 
punched cards were used for data recording. 

A series of questions was also given to the students, teachers and school principals rn 
questionnaire booklets. The variables (representing inputs to and processed in th<» school) for any 
one subject at any one level numbered approximately some 200 to 500. The sheer amount of infor- 
nfxation collected (over 150 million pieces) makes this stqdy one of the largest ev€?r in *he field of 
Social Science. As an example. Appendix II lists the student, teacher and school variables 
associated with French. These are the variables used in every country. However, each country 
had the opportunity to ask further questions which were of specific interest for its own system. 
Hence, although this paper will present the findings of the international analyses, each National 
Centre will be publishing a national report uting p fuller" set of data (three examples of national 
"reports are those of Australia^/, Japan^/ and Sweden 4/1. A forthcoming issue of the Compara- 
tive Education Review will contain a series of articles from most National Centfes, each concerning 
a special national analysis undertaken on a current national problem in education. 5/ 
c • ■ 

1/Peaker, G.F., An empirical study of education in twenty-one countries : 
A technical report . International Studies, in Evaluation, VIII, Almqvist & 
Wiksell, Stockholm (in press). ' 

' . 2/Rosier, Malcolm J., Science achievement in Australian secondary schools , 

Australian Council for Educational Research, Melbourne, Australia, 1973. 

3/Shimada, S. et al, , International survey of science education : Report of 
Japanese National Commission of lEA , Vol. I, Results of Japanese Survey, 
NIER, Tokyo, March 197.3. 

4/Hus^n, T. et al . , Naturorienteranda Umnen , Almqvist & Wiksell, - 
Stockholm, 1973. 

5/Postlethwaite, T. Neville (ed. )^ Comparative Education Review, 
Vol. XVIII, No. 2, June 1974 (in press) 
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The ainns of the research were to identify thoi^ factors accounting for differences between 
countries^ between schools and between students. The technique used was a cross-sectional survey^ 
albeit at three different levels. This« therefore^ innplies that this is a description of education as 
it is an not as it might be. It also nneans that there are no direct nneasures of past events - surrogate 
measures had to be used. A longitudinal study would have produced such measures^ but for various 
reasons the I£A felt that the time and money required for such a study were beyond its reach. The 
input and process variables data were collected from students and school personnel. The limitations 
of questionnaire procedures are that^ on the one hand^ the responses may not be accurate and« on 
the other hand^ may provide only an incomplete and often indirect indication of the phenomenon in 
which one is interested* These constraints are important to note and bear in mind when interpi^eting 
the results. 

Before passing to the results^ it should be mentioned that the fact that this study has been 

.conducted in twenty-three nations with fifteen different languages of instruction is in itself something 

of a tour jde force . In spite of translation problen^s and the difficulties of planning and executing such 

a project, not to mention the research skills required by so many people in so many countries, it 
► " 

has been possible to measure outcomes in a reliable way and to make meaningful comparisons. 
There was a certain variation in the quality of the research competencies possessed by the partici- 
pating centres and the standard form of research proved to be veiy effective in raising the standards 
,to the highest common factor. * It aliso allowed each nation's educational system to be seen in an 
international context. 
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L SELECTED. FINDINGS t 

/ 

1, Between countries 

With the liniited nuniber of observations of country nnean' scores it is not possible to undertake 
between- country niultivariate analysis. However, differences in nneans and distributions on single 
variables are of interest as well as bivariate relationships. lEA has always stressed that it is not 
conducting a 'cognitive olynnpics' and that great care should be exercised when connparing country 
mean scores. First, although international tests have been constructed and each country Was happy 
to use them they were not 100 per cent valid for each country and some countries may well stress 
some objectives which were not tested by the tests. Secondly, the proportion of an age -group in 
school varies from country to country even at the same population level, hence making comparison 
difficult. This being said, some of the test scores between countries are of interest. In what 
follows, four different modes of presentation have been used\ For Reading Comprehension, the 
pass-fail rate on particular items is given, For Literature, a profile of four diffierent aspects of 
literature is given. In Science, the increment in General Science performance froip one population 
level to another ia presented. For the other subjects, means and standard deviations are presented. 

Reading 

Perhaps the most dramatic finding is the yery large difference in performance between developed 
and developing nations. If we take Reading,^ we find that differences among developed isountries 
are fairly modest. However, with complete consistency, the thrjee developing countries fall far 
below those thkt have a relatively high level of economic development and a long-standing tradition 
of universal education. " . 

To exemplify this somewhat, consider one selected passage from the battery of reading tests , \« 
for 10-year^olds: ^ 

Population I (Passage and itemsV 
One of the most interesting birds I have seen is the Indian Tailor Bird. It is-a small olive green 
bird that does not look at all unusual, yet it has a most unusuaj way of making its nest. -The birdls ' * 
work together in pairs. First they find a leaf, the right size, and make holes along the edges with 
their beaks. Through. these holes they thread grass. One bird pushes the thread from the outside, 
while the other bird sits in the nest and pushes it back until the edges of the leaf are sewn together 
to nltelB a kind of bag, still hanging on the tree, in which the Tailor Bird lays its eggs* 
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What does the Tailor Bird use in place of thread^ 

- grass 

- string 

- spider web 

* thorris 

The Tailor Birds are interesting because they: 

- are snnall and olive green in colour 

- live in pairs ^ 

- niake their nests in a special way 

- fly very, fast , ' 

The Tailor Bird fi^ot the nanie because it: - ^ 

* is a snnall bird , 

- looks unusual 

- can sew • 

- has a beak shaped like a needle 

The Tailor Birds niake their nests:' 

- fronn 'leaved 

* in a hdle in a tree 

- in the tall grass 

- wlth^'a lining of grass 

The person who wrote about Tailor Birds was trying to; 

* gtv^ you Sonne new information , « 

-'tell you a story ^ • ' • i;*. 

- get you to «hare his feelings ^ 

- keep you guessing on how the story will come out. 



Clearly, the little paragrapt shown here- is >%it a leyel thA presents difficulty to a substantial fraction 

of lO-j^r-olds in every country, Consid^ering all fourteen countries and all five items, the typical 

failure percentage rCins between 35 per cent and 40 per cei^t (Table 2). The median percent stge goes 
/ , ^ 

as loir as 26 per cent in Finland, but is 43 per cent in Chile, 58 per cent in India and 65 per cent in 

Iran. When one considers that rahdpm marking would be expecte'd to giv»w)ly 75 per cent of error 

on these four-choice items, it bebomes clear that even this passage is pushing the limit of compe- 

ence of most 10-yeap-olds in these three countries. 

Similar results were obtained for Populations II and IV. In 6ne passage for Population II the 

* . ~. * • 

medi.aQ erfor rate was approximately SO per cent 6ut in Chile, India and Iran the rate was 47, 63 and 
61 per Cent respectively. For "Populatiorf'IV for one passage.itiejn'^dian error rate wfts 25 per cent 
but for the above-me* ^d developing countries was 44, '66 and §8 per cent respecrlvely (se€ pages / 
133-139 in the Rfeading Comprehension Repd'rt) • * , •* ty' o'*, . 



\j Thorndike, Robert L. , Reading Comprehension education in fifteen " 
countries : An empirical study . International StudieiS in EValuati6n, III, 
Almqvist & Wiksell, Stockholm, 1973. 
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■( r 

\ 

*» m 4 ^ 

Table 2. P ercent faiUhg - Population I 



Country ' 




Jtem* 1 


Item 2 


Item 3 


, Item 4 


Item 5 


Belgium (Flemish) 




5 




90 




40 






' * 12 


9 7 

6 1 


CO 


9*7 


on 
OU 


Chile 






*X O 


Do 


• -t i 


f bU 


England 




10 


O 1 


•^7 

« of 


97 


* '*7 






1 D 


36 


26 


22 


31 


Hungary 




8 


40 . 


32 


• ^'39 


48 


India . 




16 


-66 


58 


54 


61 


Iran 




56 


75 


*65> 


56 . 


66 


Israel 




12 


49 


42 


46 


56 


Italy 




8 


• 32 


80 


23 


32 


Netherlands 




8 


28 


48 . 


41 


' ,38 


Scotland 
Sweden ^ 




11 


38* 


-35 


30 


38 




10 


35 


39 


V 27 


36- 


U.S.A. , 




13 


35 


46 


38 


33. 



A further indication of the base level of competence in the developing countries is provided by error 
rate on the first nine items of the test of reading speed. These are items such ^st 

"Peter has a little dog. The dog is black with a white spot on his back and 
'one white leg. The colour of Peter's dog is mostly (i) black, (ii) brown, or 
(Ui) grey. " 

The percentage of wrong answers on items such as this is as follows: 



Country 

Belgium (Flemish) 

Belgium (French) 

Chile 

England 
^ Finland 

Hungary 

India 

Iran 

Israel 
^ Italy 



10 -year-olds 
8 
9 
26 
9 
11 
20 
36 
52 

11 



14 -year-olds 
'3 

. - 2 
16 
4' 
8 
8 
33 
20 
9 
9 
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Netherlands 8 ' 4 

New Zealand . . ^ (not taken) . . 4 

Scotland . * ' 10 . . . " ' 3 ^ 

Sweden V. • ^ '2 

-U.S.A. ' 11 • ; . 4 

In the European countries, a typical error rate on the'se items is about 10 per cent for 10-year*olds 
and 4 per cent for 14-year-olds. With these values, one nxust coi)trast percentages of 36 and 52 
for 10-year-olds of the developing countries (Chile, India, Iran) and 16 per cent, 33 per cent and 
20 per cent for the 14-year^old8, Admittedly, this material was given as^ a speed test. But it was 
also given as a reading test! If a substantial proportion of the students in a school system have re^l 
difficulty in reading these materials, one must question whether any^ more than a minimal level of 
literacy has been achieved in that school system. ^ "* 

Literature 

Literature was only tested by Populations II and/ IV in ten countries. Figure I presents a country 
. profile of four scores: 

- Coniprehension is a reading comprehension score of literary pros6. 

- Interpretation is an interpretation (primarily inference) score of literary 
prose. . \ . 

y I, 

- Transfer is a measure of the extent to which a student brings his literary • 
experiences to bear on the rest of ^is life4nd vice-versa. ' • 

- Interest is a measure of the amount of interest a'student, expiressed in 

9 ' • reading literature. . . . . ' 

It can be seen that the achievement scores (that is. Comprehension and Interpretation) are lower in 
Chile and tran than in the participating developed countries. The Tr&nsfer and Interest scores are 
presented as an illustration of non- cognitive outcomes of an educational system. However, it should 
, be pointed out that probably the most important measure in Literature was the response -preference 
meakure, where each student responded to the measure three times - once in a questionnaire about 
literary works generally and twice as re^pon^se to two literary extracts. 

The response-preference measure sought to determine the characteristic way by which a 
group of students approached a ^particular literary work or literature in general. For example, 
some students might be mox^ concerned with the historical aspects of the work; other students might 
be more concerned with its aesthetic form; still others with its theme or meaning; still others with 
its affective value. Students in both Populations II and 'IV were asked to indicate the questions they 
thought most important following a reading of each of two short stories and with reference to lite- 
rature in general. ~^ • ^ . • . 

12 
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Figure 1. Cross- national score profiles : standard scores 
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A multivariate analysis of variance Indicated that students* response preferences were 
strongly influenced by the nature of the selection that they read, by their age (that is, by which 
population they were in) and by their, country. This analysis as well' as an examination of the 
response preference patterns for the Population II and Population IV students led the committee 
to Conclude that a student's response is determined by the nature of what he reads apd by the 
educational system. During the years of secondary school, students in all of the countries examined 
appeared to have learned to focus their critical attention to jcertain aspects of literary works. A 
icofxlparison of the students* response preferences with those of the teachers of literature in the 
schools shows that the students learned to approximate the response patterns of .their teachers. The 
study of response preference also- indicates quite clear similarities and differences between coun- 
tries as the students in those^countries approach ^literary work. There emerge certain quite clear 
cultural patterns. Students in Chile, England and New Zealsind are similar to each other in their 
interest in formal aspects of -literary works. Students in Belgium, Finlcmd and the United States of 
America are marked by their interest in the thematic aspects of the work. Students in Italy are 
primarily interested in the historical aspects of the work. ^ , 

Science 

Because there were anchor itenfis between the tests for the various populations it was possible to 
bring the population test scores on .to a common scale and examine the increment from one population 
level to another in Science, as measured by the lEA tests. Figure 2 presents the increases. Again, 
the developir. J countries are presented separately - note the different position of the zero^ point. The . 
•retentivity* figure given next to each country's name is an estimate of the percentage of an age-group 
^ still in school in the pre -university year grade. It is also at Population IV level that the differences 
in interpretation of what this population was has operated to enhance differences between countries. 
Note that Australia aiad New Zealand did not test 10-y ear-olds and Japan did not test the final year 
grade group. France is missing from the table since it only tested one population, namely "la classe 
terminale" and hence no increment measure was available.^ The left-hand side of the white bar 
represents the scaled mean score at age 10, the right-hand side the scaled mean score at age 14 and 
the right-hand'side of the crossed bar represents the scaled mean score of the pre -university ye^ir 
group. 

In the Science report, the authors have broken down the total Science score into a series of 
sub-scores: earth ^ience, physics, chemistry, biology, and practical science on the one hand, and . . 
on the other hand a classification of educational objectives such as functional knowledge, comprehen- 
sion of a scientific principle, application of knowledge for problem -solving and analysis, synthesis 
and evaluation. !_/ The coxmtry-l^-country profiles of such sub-scores help to identify the relative 



2/ Bloom, B.S. (ed. ), Taxonomy of educational objectives : the classification 
of educational goals. Handbook I, Cognitive Domain, David McKay Company 
Inc., 1956. 
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strengths and weaknesses of the students perfornnance. Further to this, the iteni-by-iteni analyses 
of the Science itenns show not only the percentage of the population answering the item correctly 
(which can in a certain sense be construed as a criterion-referenced test) but also the types of 
wrong answers typically produced* Both the profiles and item analyses are extremely useful feed- 
backs to the curriculum developers as a check tb what extent performance matches the global and 

detailed objectives set for Science education in a particular country. 

r 

Figure 2. Increase in level of performance in Science from the 10-year-old 
level to the terminal secondary school stage 



Coumnr 



Scito in hundndtfw e« a itmkid <kvi«tion 
0 ' 100 




Inn 9 ^ 
Thailand 10 

India 14 

J 

oa% ie 



jmiiiii 



□ Increase from Population I to Population II 
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Each participating research centre will be carrying out a set of special national analyses. Where r 
different curricula exist for different school types or regions within a country, such profile and item 
'Analyses will be important as feedback to the curriculum developers. In a centralised syst&m of 
education the feedback links are, of course, ^easier to forge or maintain than in a decentralised 
System but it is hoped that all research centres will create the necessary links, however complex 
this may be. . ' " - 
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f» • ■ 

Experience suggests that boys show a greater "interest in Science" than girls and perform 
better on Science tests. These impressions receive powerful support from the IE A study. These 
differences exist across all countries and the differences increase as the students grow older. Thus« 
at ten years of ^age^ boys performed on average about one»quarter of a standard deviation higher 
tham girls^ half a standard deviation at 14 years of age, and three-quarters of a standard deviation 
higher in the terminal year of secondary education. Boys are more strongly attracted to the 
Physical Sciences and girls to the Biological Sciences. The score differences in Biology were less 
than in the other branches. This is clearly a problem which deserves earnest attention in the near 
future in all countries. 

French as a Foreign Language _1/ ^ 
Table 3 presents the means, standard deviations and jnombers of students for the four skills i^ French 
for each participating country at both Populations II and IV levels. When examining these scores, it 

I- ' , 

. Table 3» Means/ standard deviations and numbers of students for Sv. es 

in four major skills oft French - 



Reading Listening : Total Writing Spfeaking Fluency 
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England 


9. 3 


9. 2 
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27. 3 
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8. 8 
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32.5 
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U.S.A. 


7. .6 


7.5 


4 


177 


6. 1 


7. 2 


1 


927 


46. 6 


27.7 


2 


329 


59. 


2 
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5.2 




702 ' 


' 27.9 
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972 
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32. 3 


237 


Sweden 


19.6 


9.4 
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755 


19. 3 


10.2 


^1 


742 


67.4 


28. 2 


1 


670 


107. 


9 


41.8 


186 


U.S. A, 


17.5' 


9.5 


'3 


069 


13.8 


11,5 
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370 


57:6 


27. 3 


1 


776 


91. 


8 


37. 1 


178 



1/ Carroll, John B., International Studies in Evaluation. V : Fr ench as a 
Foreign Language in Seven Countries. An Empirical Study. Almqvist and 
Wiksell, Stockholm (in press). , . . 
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is important to bear in nnind what CarrolU/ has written elsewhere. The national mean scores 
were in general highly correlated with the average number of years that French had been studied 

-and-as-a-eor^Uary-ta-thi«-werai:e-score5T^-l^^ grade in which the study 

of French began. j 

English as a Foreign Language2 / 

Table 4 presents the means, standard deviations and numbers of students for the participating 

countries in the four major skills of Reading Comprehension, Listening Comprehension, Writing 

and Speaking Fluency. Blanks denote non-participation. The large standard deviations for 

„ Population II Reading are noteworthy. 
- Table 4. Means, standard deviations and numbers of students for scores 

in four major skills of English 

Reading Listening Total Writing Speaking Fluency 
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10. 6 
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6.5 
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8. 3 
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7. 3 
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119 


Israel 


,27.4 


12. 9 


604 




















Italy 


20.2 


12. 6 / 


324 


14.7 


7. 6 




314 


18.9 
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7. 2 
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.9.0 
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Thailand 


19.7 


9. 1 
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ll Carroll, John B. , Factors accounting for between- student, between- school 
and between-nation^ differenced in performance In French as a Foreign Language . 
A presentation at AERA Convention, New Orleans, Louisiana, February, 1973. 
2l Lewis, E. .Glyn, International Studies in Evaluation, IV ; English as a Foreign 
Language in Ten Countries . An Empirical Study. Almqvist and Wiks^U, 
Stockholm, (in press). 
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Civic Education 

As can be seen from Appendix I, there were many outcomes of Civic Education measured - attitude 
scores in addition to the Cognitive Test Total and subscores. In Table 5, means for Civics Cognitive 
Test Total are compared for different countries within population. (Comparisons between age 
population groups are not justified because the Cognitive Tests were composed of different items for 
each population. ) The phenomenon noted in the other subjects, of considerably lower scores for 
less developed. nations, holds.true in Civics for Iran. The other countries' mean scores are fairly 
closely clustered; it is interesting to note however that at both the Population II and Population IV 
levels, students in Ireland achieve relatively low mean scores* The anti-authoritarianism scale, 
-which is derived from a series of attitude items concerning democratic valqes, administered^in the 
same form at each age level can be compared across F>opulatiOn. Anti-authoritarianism is a good 
representation ofcthe first major attitudinal factor'derived in factor analysis on the 14-year-old \^ 
(Population II) students and was used as a criterion in the regression analysis (to be briefly sumnia- 
rized later in this paper and completely presented in the Civic volume) 1^/. Although the between- 
country and between-population differences are interesting for speculation, of more interest are 
patterns of attitude scale difference which are too complex to be examined here. For example, in 
some countries high anti-authpritarianisnt^is combined withra low sense of citizen efficacy; in other 
countries both types of attitudes are low. . 

^ (!)ther between- country differences ^ . 

It is not ^ssible to carry out a multivariate analysis between countries (with so few degrees of 
freedom) to Identify the factors associated with differences between national mean, scores. However, 
certain differences in certain factors are striking and provide interesting leads. Four examples are 
given below: - 

(i) Opportunity to learn in Science 
In each school in which Science was tested the Science teachers in that school were asked collec- 
tively to rate each iteni"5:n the Science tests as to the appropriate percentage of students in the 
target po^ulktion who had had "the oJ)portunity to learn" the substance tested in the item. The 
scale was: 

- ali students 4 ' , 

- more than 75 per cent of the students 3 
" . between 25 and 75 per cent of the students ^ 2 

- less than 25 per cent of the students 1 

- none of the students ' 0 



1/ Farnen, R.F., Marklund, S. , Oppenheim, A. N. , and Torn^, Judith V., 
International studies in evaluatio^n, VI : Civic Education in ten countries - - 
an empirical study , Almqvist and WikselU Stockholm (in press). 
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Table 5. Means, standard deviations and numbers of students for scores 
in three outcomes in Civic Education 
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Despite the crudity of the measure it shows a considerable difference between countries when 
summed to a national level. Figure 3 presents graphs of opportunity to learn against Science total 
test scores for Populations I, II and IV. The relationships are -striking. However, one cannot 
help wondering if, for example^ in Population 11. the tests are more appropriate for Japan than 
for the Netherlands and if the Netherlands* students are learning other things in Science not 
measured by the Science tests. This is for thet curriculum developers to decide, but it should be 
pointed out^that the test construction was a lengthy andrdetailed exercises involving Science educators 
in all countries. All countries expressed themselves as agreeing that the tests were a reasonable 
general measure of Science as'taught at the particular level. The lack of relationship for 
Population I may be attributed to the unstructured curriculum in Science in most countries at that 
stage of schooling. 

(ii) Social bias ^ 

There has been much discussion recenUy concerning the equality of participation, in the whole 
school system, of students from all social classes. Each research centre utilised its own social 
class set of categories. (This differed from the lEA mathematics study, 1962-1967, where one 
international set of categories was used. ) Therefore, although the categories are not exactly the 
same from country to country and therefore it is difficult to make comparisons between countries, 
it is possible within countries. Table 6' presents the percentage of students for each population 
in selected centres occupational categories for seven c6untri,eQ. <- 

The progression from population to population is of interest and the difference between 
England, the Federal Republic of Germany and the Netherlands on one handi and Finland, Hungary^ 
Sweden and the United States of America on the other hand, is striking. It should be recalled that 
the Federal Republic of Germany only tested students in the Qberprimat*ia in the Gymnasium . 



Table 6. Percentage of students for each population in selected categories 
^ of father's occupation (Science^ "Reading Comprehension and 
Literature) . ^ 
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Figure 3. Graphs of opportunity to learn against Science Tot al Test Scores 
for Populations I, II and IV 
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(iii) Does more mean worse ? 
It is often argued that "more means worse" in the sense that the higher the proportion of an age-group 
allowed into the final year of schooling, not only will the average achievement be lower but also the 
achievement of the €lite students will be lower. 

Taking Science as an example. Figure 4 presents the average mean score of those in full-time 
schooling in the pre-university grade - the bottom Une (the verUcal axis gives the score; the figure 
under the bottom line for each country represents the estimated {Percentage of an age-group in school; 
Inence Sweden with 45 per cent of an age-group In school has a higher average for that group than 
France with only 29 per cent of an age-group in school); the average of the top 9 per cent of an 
age-group (since the Federal Republic of Germany only has 9 per cont of an age-group in the 
Gymnasium) - the line second from the bottom; the average ^of the- top 5 per cent - the second line 
from the top; and the top 1 per cent - the top line. The comparisons between countries are fasci- 
nating but it is clear that the differences in score shown by the top 1 per cent, or 5 per cent or 9 per 
cent are not related to tjie differences In percentages retained in school; "more does not nnean worse". 



Figure 4. Science mean scores of top 1 per cent, top 5 per cent, top 9 per 
cent of an age-group^ arid overall groiip 
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(iv) Single varialfale s 

For all questionnaire itenns (student, teacher and school questionnaires) means, standard deviations 
and frequency distributions were produced for every population in every country. Such extensive 
information is typically not collected by Statistics Divisions of Ministries of Education. For each 
variable there is a known standard error of sampling. The data collected on the hundreds of 
variables used in^ the lEA^ survey _constitutes a mine of information not only for national Ministries 
of Education but also for international agencies such as Unesco and OECD, Furthermore, a compa- 
rison of the national means of a particular variable is of interest. As an example. Table 7 presents 
the means of the student-teacher ratios for each of the populations tested. 

Table 7. Student-teacher ratios by country and by population (Science, 
Reading Comprehension and Literature 



Student-teacher ratio 1^/ 
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Population II 


Population IV 


Australia ' 






19 


20 


Belgium (Flemish) 




22 


IS 


14 


Belgium (French) 




18 


f 

12 


14 


Chile^ 
England 




40' 


39 


41 




28 


18 


•IT 


Fed. Rep. of Germany 




33 


27 


20 


Finland 




19 


20 


22 


France 








16 


Hungary 




20 


20 


♦ 17 


India 




,36 


27 / 


26 


Iran 




46 


74 


72 


Israel 




20 


15 


14 


Italy 




20 , 


16 


17 


Japan 




27 ' 


i 22 




Netherlands : 




31 


IS ^ 


17 


New Zeaxand 






22 


22 


Scotland 




27 


17 


' 17 


Sweden 




17 


14 


15 


Thailand 




24 


24 


24 


U.S.A., 




26 


20 


, 20 



1^/ Data reported by schools. This ratio comprises the total number of 
students in a school divided by the total number of full-time teachers 
(two half -.time equal one full-time, etc. ) in that school. It does not 
include teacher aides or. technical assistants. 
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2.- Within countries 

Within countries the student score in the various subject-matters was used as the school outcome 
(dependent variable) and the input and process factors as the independent variables in a regression 
analysis whereby an attempt was made to identify the importance of groups of variables in accounting 
for variance among students in their performance. Certain groups of variables were formed. The 
details of how these groups were formed, both through the sieving and compositing of variables, is 
reported in detail in the various reports and particularly in the technical volume. 1^/ For each 
subject area there were, as has been mentioned above, some 500 independent variables describing 
the students home background, their previous schooling and current schooling, the teachers and 
their teaching, the school principal, the organization of the school, its size and budget, etc. 

CUistefing of variables 
The miajor groups (or blocks) of variables used were: 

1. Home background .^ A weighted composite of father's education, mother's education, father's 
occupation, number of books in the home, usie of dictionary^in the home and size of family. (Xhere 
were minor deviations in this composite between subject areas, and in one countzy, Sweden, the 
first tVo variables had to be omitted because of ambiguity in- the Swedish translation of the questions). 

Sex, age . (Age was entered in Block 3 for French and English. ) 

2. Type of school and/or programme in which the student was then enrolled. (The elements of 
each variable were criterion scaled against Reading, Word Knowledge or Science in order to form 
the variables type school and/dr type of programme. ) This variable was Intended as a surrogate 
(perhaps a weak one) for previous schooling. 

3. School and teacher variables (for Science, Reading Corr^rehenslon, Literature and Civics). 
Time variables , e. g. years studied subject, cu^rt-ent grade, gracje beginning study of subject 

Knglish). > 
4. , Kindred variables , e. g. expected occupation, education, attitudes, etc. , (for Science, 
Reading Comprehension, Literature ^ind Civics)." 

School and teacher variables (for French and English). (In French and English the kindred 
variables formed a fifth block.) 

Typically, two further groups of variables were used but not consistently from subject to 
subject^ The Word Knowledge scores (on an antonym > synonym test) were used and the Reading 
scores where Reading was not the criterion. . 

Within countries two major types of analyses were undertaken - a between- student analysis 
and a between- school analysis. The between- school analysis was .undertaken for Science, Reading 
Comprehension and 'Literature only. In each case the sieving of independent variables (particularly 



1^/ Peaker^^ G.F;, An empirical study of education in twenty- one , countries ; a 
technical report, 'International Studies in Evaluation, VIII', Almqvist and 
Wiksell, Stockholm (in press). • 
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the instructional, school organizatipnal and kindred variables) was based on the partial correlation, 
after home background and/ or type of school,' type of programjpie had' been partialled out, exceeding 
twice the standard error of ^annpling and the importance attached to a particular variable by the 
subject committee. In this way, the effect of school aod teacher variables could be ;iudged after the 
calibre of input of students to^the school had been taken account of. 'Cases ^f detect^ high multi- 
coUinearity were 'solved* by compositing the variables in question. Where the school and teacher 
variables w.ere basec^.qn fewer than 50 observations (i.e, 50 schools) there is a probleni of stability^ 
of estimates. The populations in this category arable two Belgiums (Population^ IJ and IV), Iran (all 
Populations) and the Netherlands and Hungary (Population IV), It should also be pointed out that, 
where a variable had 20 per cent or more missing data, it was dropped from the analysis. This was 
the case for all data relating to the sch&ol budget. However, there are countries where these data 
are complete and further analyses should be run whereby thi? effect of differential school budget 
aspects are examined for their effectiveness after at least home background has been partialled out/ 
Some variables (e.g. class si^^,'^ laboratory class size) were omitted because they were curvilirieaY. 
Th 6^ separate analyses of these were never unde^t^l^n because temporal and financial constraints, 
and clearly such ahalyses have high priprity. . .. V / 

Table 8 presents as an example the surviving variables t)y b]Lock for New Zealand Science 
Population II between- school analysis. There ace forty single variables which with compositing form 
'hirty variables. This was a typical number of variables fbr the final regression analyses, - , 



Incremental variance 



Appendixes III to VIII present the incremental variance for various blocks of variables, a^ well as 
the total variance accounted, for in each subject ar'ea in each population for the between -student 
analyses. Table 9 presents a summary of the block Variances and total variances account. 

Th^ percentage of total variance acc9Urited for varies frOm a low of 11 per' cent in Iran for 
Population IV Science to a high of 79 per cent in Germany for Population II English. The average^ ■ 
percent toial variance accounted for was 38^9 per cent*" which indicate? that there is'a great deal'^of 
unaccounted variance and which implies work for educators to ide^itify and measure pther. factors 
not yet included'in the lEA analyses. The total variance accounted for is notably lower in developing 
countries. . ^. , . . \ • , 

^ The home background variaBles vaty from 1 per cent to 30 per cent, with an average of 
11. 5 per cent, which r&ise3 the question of to what extent are the appropriate home variables being 



tapped in some countries. , ■ * ' 

Learning conditions vary from a lo.w of 1 per cent to a high of 52 per cent. The average is ^ 
10 per cent across all populations in all subjects in all countries. The percentages are clearly lower 
for Reading Comprehension, Literature and Civic Education than for Science, French and English. 
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Tablfe 8. Variables suirviving to final bet ween- school regression analyses 
in New Zealand Science Population ir 



Block 1 Father's occupation 
Father's education 
Mother's education . 
Us^ of dictionary 
Books in home 
Family size 
Age 
' Sex 



Home background 



Block 2 Type of school 

• Type ol programme 

Block 3 Percent male teachers in'school 
No, of laboratory assisi.tants 
' Sex of teacher 
'OpF>ortunity -to learn^ 
School environment i 
Total hpmev^ork per week 
Total science homework per week 
Taking sci^^nce 
. Total years study of science 

' Total hours study of science 

Principal's teaching experience 
Total enrolment. 
Decision-making - syllabus * 
Admission criteria - residence 
- exams 

Economy of school region ^ 
Methods - audio-visual 

^ - field trips 
Science expeditions 
In-service biology course 
Total in-service tradning 
-> Grade in school ; ^ 

Science teacher -training 



Homework composite' 

c 

Study of science 
composite 



Block 4 Interest in science ) 
Science in work ^ ) 
Importance of maths ) 
Reading science/technical 
^ Reading science fiction 
Heading science articles 

Watching TV - science- related programmes 



Attitude composite 
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Table 9.. Averages anct ranges of Ml countries of contributions by selected 
blocks to total variance in the between-student analyses for each 
subject 



Population I 



Population II 



Population IV 



Average Range Average Range Average Range 



Home 'background (Block 1) > 
Science 

Reading Comprehension 
Literature . 
'Civics (cog, ) 
French (reading) ? 
English (reading) 
* Previous Schooling (Block 2) 
Science 

Reading Comprehension 

Literature 

Civics 

French ' 
English 

' Learning conditions (Block 3 for Sc/,ReC. 
Lit. and Civics, Blocks 3 and 4 for Fr. 
and EngV. ) 

Science i 



Beading Comprehension 
'Literature 

Civics _ . 

French \J 
.•English 

Total variance 2/ 
Science 

ileading Co Apprehension 

Literature 

Civics 

French 1^/ 

English 



11 


1 - 27 


16 


3-29 


13 


3 - 25 


14 


1 - 25 


16 


1-27 


8 


1-18 






IS 


*j 5-25 


9 


4-17 


- 




16 


9-20 


10 


5 - 18 






15 


3-15 


4 


1-16 






14 


1 - 26 


7 


1 - 16 


1 


0-14 


6 


0 - 17 . 


6 • 


0-26 


1 


0-9 


10 


0 - 21 


7 


0-19 






8 


0-14 


3 


0-7 






11 


•5-20 


5 


0 - 15 






22 . 


1-22 


3 


0 - B 




f ■ 


20" 


0-52 


9 


0 - 31 














8 


1 - 21 




4-23 


15 


4 - 41 


6 

{ 


2 - 18 




3 - 10 


5 


2 - 15 






. 7 


3-12 


4 


- 1 - 6 






11 


3-18 


5 


1-9 








12 - 24 


19 


10 - 52 






is 


8-27 


17 


4 - 29 



27 
28 



14 - 36 
18 - 39 




36 
39 
55 
59 
45 
60 



17 
20 
36 
55 
28 
47 



55 

51 

66 

62- 

65 

7.9 



39 
25 
37 
39 
43 
44 



11 - 63 
13 - 47 
28 - 53 
•28 - 57 
27 - 61 
22 - 75 



1/ This is'with Readifig as the criterion. For Listening th?" percentages accounted for are greater 

2/ The total variance is always unity or in this case unity x 100. Both in this table and in Appendixes III 
to VHI, it will be noted that the sum of the percentages of the three blocks for ai^ one subject do not 

..total to the percentage in the 'total variance' block. This is becausj^the 'total variance' in each subject 
is the sum of the first three blocks plus a kindred variables block and typically oiie or two other blocks 
whi<^ vary from subject to subject but which usually include a Word Knowledge score, which might be ' - 

-r^^rded as a partial, surrogate for 'intelligencD' and .'previous experiences' not' measured in Blocks 1 
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Furthermore, the percentage in these last three subjects rises with the school population being 
examined. Let us be clear on what these percentages ar«. In a cross-sectional survey of this 
kind we are saying that after the home background of children has been taken account of, the 
differences between schools (as they now exist) in the learning conditions they pre /ide are associated 
to a considerable degcee with differences in performance between students on the criteria under 
- consider Aion. In some cases, the learning conditions are two -thirds of the total variance accounted 
for. , 

•Rut then why should these differences between learning conditions be more important for 
Scienc , French and English- than for Reading Comprehension, Literature and Civics* Is it that the 
first three are more school-oriented in that specialized knowledge i;^ being learned where there 
exists little knowledge in the home and therefore little backing? Or is it that Reading Comprehension 
is not systematically: taught in schools once the decoding and encoding of the mechanics of reading 
have been accpmplished? As a corollary to this, would we expect differences in learning conditions 
to be greater amd more associated with student performance if it were taught systematically. Is 
the same true of the sort of civifcs and literature tested by the E3A tests? Several alternative expla- 
nation^ can be conceived but What do the educators in th^se subject areas think? 

Incremental variance between ^schools 
For the details of the between-school analyses, readers are referred to the main publications 1/, 
(between-school analyses weire undertaken for Science, Reading Comprehension and Literature only) 
but an example is given tjere for Scotland and Sweden Population II in Science. ^ 





Scotland 


Sweden 


1. Home background 


80 


8 


- age ^ 


1 




- sex 


0 


5 


2. Type of pro gram m'e 


2 


28 


3. School 


3 


16 


4. Kindred 


_2 


_6 


Total - ' r ' 


89 X 


63 



X Figures making up this table have been rounded. 
When the home background of students is aggregated to the sc5;iool level (neighbourhood effect? ) the 
difference in the percentage of variance accounted Jor is dramatic between Scotland and Sweden. In 
Scotland the society is clearly stratified in its housing and district scHool provision according to home 
background whereas the difference bet^ween neighbourhoods in Sweden is minimal or another way of 

_!/ international Studies in Evaluation, Vol. I: Science education in nineteen 
^ ^ countries, an empirical study by L.C. Comber and John P. Keeves; 

Vol. II: Literature education in ten countries, an empirical study by Alan C. 
Purves; Vol. Ill: Reading comprehension education in fifteen countries by 
Robert L. Thorndike, Almqvist and Wiksell, Stockholm, 1973. 
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saying this is that schoal neighbourhoods are more homogeneous in Scotland, With the home back- 
ground accounting for so much variance in Scotland, clearly there is little variance left whereby 
differences between school processe^^ (or learning conditions) can account for further differences. - 
On the contrary in Sweden, with its socially relatively homogeneous society, the differences between 
schools as they are run accounts for two-thirds of the total variation. It should however be recalled 
that the two variables, father's/ mother's education had to be omitted from the home background 
composite,, but it is unlikely that this would have raised the 8 per cent to more than 10 per cent. 

< . • ■ ' 

Learning conditions variables 
In those cases where differences in learning conditions between schools and students exist, after the 
calibre of input of students to those conditions has been taken account of, it is important to, identify 
the factors important in the learning conditions. To g^^erallze, the following factors stand put 
tas iniportant: 

Time , i.e. years of study, number of hours instruction and homework per week prpve to' 
be important. Unfortunately, however, the EE A study did not examine the trade-off. If; for example, 
a student who spends more time on Science performs better than one who spends less time, how does 
the student who spends less time perform in some other aspect of school behaviour on which he has 
spent more time than the other student. The study of multiple outcomes for students and schools is 
clearly important in future research of this kind. 

2. The ' opportunity to learn ' (or in more general terms the curriculum) is important in that 

the more a student is allowed to learn the subject, the more he will learn. This is in general a 
powerful variable and the detailed study of this will be extremely useful to curriculum planners. 
3.. The amount of teacher tertiary education plus pre-service teacher training is important 

for student performance, particularly in the hi^er grades in school. 

4. In foreign languages", the extent to which the students and teacher speak in the foreign 
language is important. This does not imply, however, that total use of the direct method is a 
panacea for ijiigh standards of performance ^in all aspects of the foreign language. . 

In French the major variables emerging from the multivariate analyses are always: time , 
aptitude , interest, and quality of instruction - in that order of importance. It should also be added 
that there was considerable variation between countries in ' teacher competence ' as indicated by such 
variables as rated competence in French and amount of time spent in a French-speaking country. 

5. Measures of classroom climate and the use of practices such as patriotic rituals in the 
classroom were associated strongly with variation in civic attitudes and to a lesser extent with the 
civic cognitive score. 

For more subject-areas at each population level det9.iled comments have beeq made about 
the learning conditions variables and, as an example, the authors of the Science report comment on 
each of the following variables at Population IV level for the between- student analysis: 
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Population IV . Grade, science study and homework, total years study of science, students* planning 
of investigation, total enrolment of school, science teachers' involvement in science curriculum 
reform, teachers* post- secondary schooling, teacher training in Biology, teacher subject association 
membership, teacher's preparation of lessons. ^ 
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IL FINAL REMARKS v 

o . . 

The worthwhileness of the results will depend to a jgreai extent on the way in which policy -makers, at 
whatever level; understand both the strengths and weaknesses of the results,. In many cas^s, this 
will depend to a large extent on the systematic links already forged or in the process of being forged 
between the lEA researchers at the national level and policy -makers in each nation. The univariates 
descriptive statistics on so many input, process and output variables for various levels in each 
school system are a mine of information for each national system of education. Some of the be^weeh- 
nation differences, as can be seen from the selected examples given earlier in this paper, have very 
pointed messages for some national systems. More will surely be provided in the forthcoming 
Passow- Noah- Eckstein report, ly . ' . 

The within- country analyses, despite the problems of correlations'between the variables in 
the final regression analysis and exogeneous variables, have identified aspects of learning conditions 
(as they now exist) which are highly associated with differences in student achievement and which 
^ suggest further experimentation and research. 7 

In a cross- sectional study of this kind, cause and effect cannot be proven, only inferred, 
and there is a strong case for submitting some of the variables shown to be important to a^strict 
experimentation. « 

lEA is in the process of establishing a data bank' of all these data. Certain parts of the lEA . 
data were never analysed (e.g. the data concerning students specializing in Science at the pre- 
university level, budget data, etc. ) tind they must surely be analysed. The within- country analyses 
were undertaken on total national seimples. It is important-to undertake f egreission analyses on^ 
specific sub-groups (whether within school type, social class or minority sub-groups,' etc. ) to 
discover the deviations from the overall findings for such sub-groups. i)Llthough many variables did 
not reach 'significance' and were therefore dropped from the analyses, certain of these are of^_ 
importance to different disciplines of education. Their inter- relationships with other variables, 
possible compositing an^ differential 'effects' on educational output should be explored. Many of 
' the variables which emerged as important can be coded. Will economists undertake cbst- effectiveness 
analyses using these data? One could proceed with such a list for several pages. 

For international organizations one of the. problems emerging from,' the IE A study concerns 
th^ types of variables on which it' would be useful (and why? ) to collect systematically standardized 
data (and how?). 
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,\j Passow, A. Harry, Noah, Harold J. , and Eckstein, Max, The national 
case study : an empirical comparative study of twenty-one educational 
systems , 'International Studies in Evaluation, VII', Almqvist and Wfksell; 
Stockholm (in press). 
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To return to the first result*presented in this paper, what sorts of pilot work can be under- 
taken to determine* how, at low cost, can one collect reliable data from national samples of children 
in developing countries where the illiteracy rate is high? What type of interdisciplinary work should 
be undertaken in developing countries to help identify the types of factors on which data should be 
collected to help 'explain' differences between students and between schools. 

Although evaluators do conceptualize in their roles as researcbei^s, -it is clear that in all 
countries the state of the art of educational theory (or in some cases perhaps the communication of 
it to the educational researchers and ^valuators) is poor, as witnessed by the low total variances 
accounted for. What types of work can be undertaken to improve this^tate of affairs? 

Finally, it is through evaluation pi'ojects of this kind that more 'hard' information can be 
collected and used to improve the educational provisions for children in the coming years. The 
knowledge explosion in measurement and evaluation techniques has been rajjid. The co-operativ6 
nature of lEA's work has helped researchers in some countries to advance. their. competency by 
twenty years in a five-year period. More such enterprises of this kind are needed. • Research 
centres and grant-giving agencies should not be deterred by the absence, in mar^ instances, of 
black and white results. Progress is always slow but decisions concerning the future cpntent and 
strategies of education must surely be ameliorated by the presence of more research findings. 



2 



ERIC 



34 



Appendixes 



Appendix I 



MAJOR OUTCOME VARIABLES - COGNITIVE AND AFFECTIVE 



1. Science 



Cognitive . ^ 

Total Science Score (different tests Populations I, II and IV, but with anchor itemsy. 

Sub- scores ; Earth Science (Population I only) 

- Physics 

Chemistry 
Biology 

Practical Science 
Functional Knowledge 
Comprehension 
Application 

Analysiaj S^nthesis^ Evaluation 

Specialist Science tests in Biology ) , . 

' Chemistry ) Population IV " 

' Physics ) g ^ 

Practical tests (Populations II and rv) " 

Test of Understanding Science (Populations II and IV) 

Attitudinal and Deaci iptive 

Interest in Science ^ r 

Attitude towards ^School Science 

Attitude towards Science in world ^ ^ . 

Description of Science teaching (textbook/ experimental) 

Description of Science teaching (laboratory work - stznictut*ed/unstz*uctured) 



2, Reading Comprehenpion 

Total Reading Comprehension Score (different tests. Populations I, II and IV, but with 
anchor item^). 

Sub-scores ; - ability to determine the meaning of a word or phrase in context 

(Population I only) 

- ability to follow the organization of a passage and to identify 
antecedents and references in it (Populaitions II and IV only) 
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- ability to answer questions that are specifically answered in the 
passage 

- ability to draw , inferences frofn a passage about its content 

- ability to determine the writer's purpose, intent and point of view. 
Word knowledge test (antonym /synonym test - different for each population but with anchor 
items). 

Reading speed test (Populations I and II only - score = item reached) 
Sub-score - error rate on first nine items. 



3, Literature 

Cognitive / 

- Total Literature Score 

Sub- scores : Comprehension 

Interpretation 

Attitudinal 

Response preference 
Transfer . 
Interest in Literature 
Degree of lilting passage 



4, English as a foreign language ^ 
Cognitive ^ . 

Reading total (different for Populations II and IV, but with anchor items) 
Listening total 

' Sub-scores : Listening sections I, II, 'ill and IV 

Writing total 

Sub- scores : Writing Quantity 

Writing Quality 
Writing Variety 

Speaking total 

Sub-scores : Structural Control 

Oral Reading 

^ ^ : Various Fluency Scores 

Attitudinal and Descriptive 

Interest in English - 

It I 

Perceived UtiUty of EngUsh 
English activities out of school 
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5, French as a foreigo language 

Cognitive ' 

Reading total (different for different populations but with anchor items) 
Listening total ("different for different populations but with anchor items) 
Writing total 

Sub-see res ; Fill-in writing test 

Writing Quantity 
Writing Quality 

Speaking total 

Sub-scores ; Pronunciation 

Structural Control 

Oral Reading 

Various fluency scores 

•» 

Attitudlnal and Descriptive 

Interest in French « . 

Perceived Utility of French 
French activities otit of school 

6, Civic Education 

^ Cognitive ^ 

Cognitive total (different for different populations but with anchor items) 

Sub-scores ; , Citizenship 

Institutions 
Processes > 
International ^ 
Economic Processes 
Social Processes 
Simple Behaviours 
Complete Behaviours 
Abstract Behaviours 

Attitudinal and Descriptive 

Democratic system difference ^coresP 

Business 
Police 
Welfare 
Unions 

Total "don't know" Score. 
Spread of -How Society Works ^ 
^Political Conflict Perception - How Society Works 
Town Council Evaluation 
Town Council Responsiveness 
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Natipnal Government Evaluation 

National Goyernnnent Responsiveness 

Anti-authoritarianisnn 

Tolerance/ Civil Liberties 

General Tolerance 

Efficacy 

Woinen's Rights 

Value of Criticism 

Attitude ''don't know" 

Influence ''don't know" 

Pressure Group Influence on Laws 

Media Influence on Laws 

Equality Attitudes * 

EquaUty "don't know" 

Avoidance of Conflict 

Active Good Citizenship 

Disengaged Good Citizenship 

Non-political Good Citizenship 

Personalised Egocentric Attitudes ^ 

Personalised Sociocentric Attitudes 

Abstract Reasoning 

Interest in Civic TV 

tMscussion with Friends 

Discussion with Parents 

Discussion with Teachers 

Total Discussion 

Unawareness of Political Opinions - * 
Agreement with Family 
Agreement with Friends 
Agreement with Teachers 
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Student, te ac her and school variables in French study 



Student 
Grade 

Father's pccupation 
Expected occupation 
Father's language 
Mother's language 
First language 
Second- language 
Sex 

Feather's education 

Mother's education 

Expected education 

Hours homework 

Place of hpmewoi'k 

Fixed time for homework 

Help with homework 

Parents correct speech 

Parents check spelling 

Dictionary in home 

Parents encourage reading 

Parents' interest in school 

Encourage culture 

Daily newspapers 

Books in home 

Hours pleasure reading 

TV and radio 

Size of family 

Position in family 

■"Like/dislike of different 
subjects 

' Interest in French . 

French activities out of 
schbol 

. - 39 



Teacher 

French in classroom 

Teach other subjects in French 

Hours instruction in French 

Sex 

Age 

Specialist teacher 
Full-time education 
Post -secondary education 
" Teaching experience 
Time teaching at this school 
Hours preparation of lessons * 
Houns marking papers 
Membership of teachers association 
Teaching journals 
Subject journals 
Attended conferences 
Assessment, standardized tests 
Assessinent^ essay tests 
Assessment^ objective tests 
Assessment, homework 
Assessment^ projects and papers 
Assessment^ variety and exteht " 
Importance of student needs 
Importance of curriculum 
Importance of textbooks 
Importance of examitiations 
Importance of next grade ^ 
Methods textbooks . \ 
Methods drill materials 
Methods individual materials 
Methods small group work 
Methods individual tutoring 



School 

Class size 

Principal's degree 

Years as' principal 

Years as principal of this school 

Years teaching experience 

Typfi^o^ community 

Museum availiable 

Zoo availab^ 

Public library available 

Concert hall available 

Opera available ^ 

Foreign language societies available 

Amenities score 

Total' enrolment, boys 

Total enrolment, girls 

Total enrolment 

Lowest grade in school 

Highest grade in school 

Mean grade of school 

Grade range of school 

Enrolment of Population I 

Enrolment of Population II 

Enrolment of Population IV ^ 

Co-education, Population I 

Co-education, Population II 

Co-education, Population IV 

Day or boarding school 

Beginning French grade 

Beginning English grade 

-Beginning Social Studies grade 

Decision- making, textbooks 

Decision-making, rules 
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Student 



Teacher 



School 



Mother studied French 

Parents interest 

Help with French homework 

Utility of French 

Aspiration in French skills 

Grade beginning French 

Entry knowledge 

Years studied French ' 

Perception spoken French 

Perception listening French 

Perception reading French 

Perception writing French 

Opportunity to speak 
French 

Time in French country 

Ease/ difficulty French 

P^erceived industry in 
French 

French books in home 

Freiquency of exposure to 
French^ books 



Methods audio and visual 

Methods field trips 

Methods lectures * 

Methods questioning 

Methods discussion 

Methods variety and extent of 
approaches 

Within class grouping. 

Part of time employed 

Teacheii-training institution 

Foreign language teaching load 

French -in other subjects 

Other foreign language 

Grade range of French teaching 

Number of groups taught 

Total time teaching French 

Mother-tongue 

Age beginning French 

Perceived listening skill, 
speaking skills reading 
skill, writing skill, and 
pronunciation skill 

Residence French country 

TertVary French in years 

Method training 

Years teaching foreign 
language 

Foreign language^association 
member 

Use of mother tongue, beginning 

Use of mother .tongue, intermediate 

Use of mother tongue, advanced 

Emphasis choices: listening 
cbmprehensioa 

Emphasis choices: speaking 
fluency 

.En\phasis choices: correct 
pronunciation 



Decision-making, cho6sing teachers 

Decision-making, conditions 

Decision-making, student selection 

Decision-making, expenditure 

Decision-making, tuition fees 

Decision-leaking, administration 

Decision-making, syllabus 

Decision-making, methods 

Decision-making, syllabus and 
methods 

Inspection 

Report to authorities 

Advice on school pr^lems 

Advise teachers 

Assess teachers' 

Role of inspection 

School operating costs 

Teacher salaries 

Non-teaching staff salaries 

Maintenance and repair 

Books and stationery 

Purchase of equipment 

Other (loan charges)^ 

Total budget 

Total budget per pupil 

Teacher salaries per pupil 

Foreign language percentage < 
budget 

») 

Number full-time staff 

Pupil/ teacher ratio 

Teacher salaries per staff 

Number of teachers male 

Number of teachers in French 

Number of teachers in English 

Number of ♦ ?hers in Social 
Studies 

School librarian 
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Teacher 



School 



Emphasis choices: reading . 
comprehension 

Emphasis choices: af^Iity to write 

Order of spoken and written • 
French * i 

Grammar teaching | 

Speaking emphasis 

Pronunciation methods 

Teaching aids* blackboard 

Teaching aids, pictures 

Teaching aids, "props" 

Teaching aids, film strips 

•■s 

Teaching aids, sound movies 

Teaching aids, phonograph 
records 

Teaching aids, tape recorder 

Teaching aids, language 
. laboratory 

Teaching aids, variety and 
extent 



French foreign language assistant 

English foreign^ language assistant 

Foreign language laboratory 
technician 

Total number of language 
auxiUaries 

Admission criteria, residence 

Admission criteria, performance 

Admission criteria, interview 

Admission criteria, examination 

Admission criteria, graduation 

Admission criteria, membership 

Streaming practices 

Scl^ool programme, variety 

T^pe of course 

J^apguage of instruction 

Student decision-making 

Hours per week schooling^. 
Pop. I, II and IV 

Weeks per year schooling 

Houl*s per year schooling. 
Pop., I, II and IV 

When spoken French intii'oduced 

When reading and writihg French 
introduced 

Class conducted in French 

Selection of French students 

Percentage study French, 
Pop. I, II and IV 

Number of periods French, 

beginning, intermediate, advanced 

Length of periods French, 

beginning, intermediate, advanced 

Level present^ beginning 

Total time French, beginning, 
intermediate, advanced 

Level present, advanced 

Native teachers French 
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Contribution by blocks to total variance from betweert- student regression 
analyses - Science 



Increment Population I Population II ^ Population IV 



Country J*^^-^ 


Block 
1 


Block 

. 2 


Block 
3 


irOxai 


Block 

1 ' 


Block 
2 


Block 

3 


X oiai 


Block 
1 


Block 
2 


Block 

3 


1 u icli 


. ^ \ 
Australia 


- 


- 




- 


16 


■ 7 


11 


39 


13 


3 


20 


44 


Belgium (iPlemish) 


4 


2 


8 


21 


8 


3 


12 


26 


- 


- 




- 


Belgium (French) ^ 


12 


0 


21 


36 


- 




- 


- 


- 


-A 




- 


Chile ^ 
England 


4 


0 


9 


26 


13 


4« 


6 


^25 . 


. 19 


3 . 


8 • 


32 


21 


0 


3 


32 


23 




7 


52 


13 


2 


41 


61 


Fed. Rep* of Germany 


8 


2 


10 


24 


18 ' 


2 


14 


34 


13 


12 


8 


39 


Finland 


14 


0 


4 


26 


22 


6 


10 


44 


25 


19 


7 


56 


France 


- 


- 


- 




- 


- 


- 


- 


16 


26 


5 


' 49 


Hungary 


8 


0 


7 


20 


14 


3 


5 


31 


11 


13 


10 


41 


India 


a 


0 


20 


29 


3 


10 


8 


24 


4 


4 


17 


26 


Iran 


6 


14 


6 


32 , 


5 


1 


9 


17 


3 


0 


4 


11 


Italy 


4 


0 


4 


14 


10 I 


' 4 


6 


24 


10 


2 


16 ^ 


30 < 


Japan 


17 


0 


1 


22 


23 


0 


• 4 


40 










Netherlands 


16 


1 


7 


29 


19 


15 


10 


49 


21 


5 


31 


63 


New Zealand 










17 


12 


8 


45 


13 


1 


, 31 


54 


Scotland 


22 


1 


5 


36 


29 


11 


9 


55 


19 


1 


34 


63 


Sweden 


16 


0 


5 


24 


18 D 


0 


7 


36 


18 


8 


20 


52 


^ Thailand 










• o 


3 


23 


37 










S. A, 


18 


1 


9 


34 


22 


2 


7 


36 


18 


9 


8 


39 


Mean 


. 11.4 


1.4 ' 


7.9 


27,0 


15/8 


5.9 


9.2 


36. 1 


12.7 


6.4 


M5, 3 


38/8 


Highest 


22 


14 


21 


36 


29 


17 \ 


23 


51' 


25 


26 


41 


63 


Loweat 


1 


Q 


. 1 


14 


■ 3''' 


0 


4 


17 


3 


0 


4 


11 . 



40 
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Contribution by blocks to total variance from betyeen-student regyessipn 
analyses - Reading Comprehension 



Increment 



Population I 



Pot)ulation II 



Population IV 





BlockBlock Block 
^^^^ 2 . 3 


Total 


Block 


Block 
2 


Block 
3 


Total 


Block 
I 


Block 
2 


Block 


Total 


toei^um yj: lemisni 




0.0 


11. e 


^Z. D 


/. 0 • 


lU. 7 


9.3 


33.7 


19 T 




Q A 


42. 0 


Belgium (Fr^ch) 


16.7 


1. 1 


' 18. 4 


38.-8 


11.2 


14.5 


10.3 


43.2 


7.2 


• 8.4 


9.0 


29^ 


Chile 


1.4 


1.2 


8.3 


22.3 


20. 1' 


5.9 


6. 6^^ 38. 6 


14.^9 


4.9 


5.1 


28.^ 


England 


22.1 


0.4 


1.5 


31.4 


27. 3 


13.7 


2.6 


50.8 


2.4 


1.8 


4.6. 


16.4* 


Finland 


17.7 


0.0 


2.8 


2^.8 


20.3 


13.4 


4.3 


46. 8 


11.5 


2.3 


2.3 


22.0 


Hungary 


18.7 


0.3 


3.8 


28.4 


18.6 


3.9 


4.0 


36.2 


6.9 


8.5 


4.8 


27.1 


India. 


1.6 


0.3 


14.9 


30.8 


1.4 


3.1 


9.7 


20.0 


4.0 


2.2 


4,7 


12. 9 . 


Iran 


8.7 




7.2 


30.3 


6. 1 


3.2 


6.6 


20.0 


5.5 


0.0' 


T.6 


14.5 


Israel 


2S.4 


1.9 


3. 6 


36.6 


25.2 


13.9 


4.3 


48.3 


8.6 


19.4 


15.2 


46.9 


Italy 


9.6 


0.2 


4.4 


18.0 


10.4 


12. 4 


3^2 


33.6 


9.6 


7.8 


5.0 


25.3 


Netherlands- 


11.1 


1.6 


4.0 


26.4 


15.5 


20.7 


3.9 


45.8 


1.0 


7.4 


4.8 


16.3 


New Zealand 










13.5 


17.6 


8.2 


46.8 


9.0 


1.6 


1.9 


19.2 


Scotland 


^23.7 


0.3 


2.3 


33.8 


26. 1 


12.3 


3.9 


50.8 


4.0 


1.4 


3.5 


20'. 2 


Sweden * ^ 


11.4 


0.3 


2.9 


18.3 


.16.1 


0.0 


2.9 


33.' 7* 


4.2 


14.0 


1.8 


24.9 


U.S.A. 


- 19.8 


2*6 


3.8 


31.8 


22. 1 


3.4 


6.5 


.42.6 


17.5 


8.0 


2.9 


33.6 ^ 


Mean 

Bluest 

Lowest 


13.5 
25.4 
1.4 


1.3 
9. 1 
0.0 


6.4 
18.4 
1.5 


28.4 
38.8 
18.0 


15.9 
27.3 
1.4 


9.9 
20.7 
0.0 


,5.8 
10.3 
2.6 


39.4 
50.8 
20.0 


8.0 
17.5 
1.0 


7.1 
19. 4 
0.0 


51*1 
15.2 
1.8 


25; 3 
46.9 
~12.9 . 
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Appendixes 



appendix V 



^ • • • ■ 

^ ^ Contribution by blocks to total variance from between- student regression 

analyses - Literature . 



Increment Population II Population IV Cognitive Population IV Interest 



Country 

— 1 


Block 

^ 1^ 


Block 
2 


Block 
3 


Total 


Block 
1 


Block 
2 


Block 
' 3 


Total 


Block , 
1 


. Block 
2 


Block 
3 • 


Total 


Belgium (Flemish) 


12.4 


13.7 


6.9 


58.0 


13.2 


'6.9 


2. 3 


44.5 


11.8 


3. 9 


6.6 


38. 6 


Belgium (French) 


18.8 


6.3 


9.0 


55.3 


4- 1 


3.2 


3. 3 


33. 6 


2'. 2 


3. 8 


5.1 


40.8 


dhile 


14/4 


6.2 


8.9 


52.7 


10.5 


3.5 


5.7 


37.7 


1.5 


0.9 


4.5 


30. 2 


England 


25.2 


11,9 


3.9 


66.1 


7.3 


0.8 


0.7 


33.7 


4.4 


8.5 


5.7 


49.7 


Finland 


18. 1 


9.7 


5.5 


55.8 


5.7 


0.3 


2.7 


28. 1 


2.7 


6. 3 


3.0 


34.4 


Iran . 


4.5 


2.4 


12. 1 


36.4 


8.3 " 


0.0 


5.7 


28.8 


4.4 


'3.0 


7.9 


23. 3 


Italy 


10.6 


- 8. 7 


3.0 


43. 8 


10. 7 


3. 3 


5.4 


40. a 


3.0. 


4.5 


5.P 


29. 8 


New Zealand 


. 15.2 


13.9 


8.7 


63.4. 


i:.7 


1.7 


2.0 


35. 1 


2.8' 


7.5 


4.2 


40.4 


Sweden 


•15. 2 


0.0 


6. 1 


57.7 


5.1 


4.4 


1.8 


37.7 


•2. 1 


5.0 


4.9 


36. 2 


U.S.A. 


18. 5 


2.4 


7.7 


61.5 


16.5 


3.4 


5.0 


53.0 


5.7 


1.2 


5.9 


35. 6 


Mean 

Highest > 
Lowest . 


15.3 
25.2 
4. 5 


7.5 
13. 9 
0.0 


7.2 
12. 1 

0 

3.0 


55. 1 
66.1 
36.4 


9.0- 
16.5 
4.1- 


2. 8 
6.9 
0.0 


3.5 
5.7 
0.7 


37.2 
53.0 
28. l' 


4. 1 
11.8 
1.5 


4.5 
8.5 
0.9 


5.3 
7.9 
3.0 


35.9 
49.7 
^3. 3 



Appendixes 



Appendix VI 



Contribution by blocks to totaL variance from between -student regression 
analyses - Civics Cognitive 



Increment 



Population II 



Population ](y 



Country .^'^'^^^ 


Block 
1 


Block 
2 


Block 
3 


Total 


Block 
1 


Block 
2 


Block 
3 


Total 


Fed. Rep. of Germany 


17.6 


6.5 


17.7 


5,7.2 


4.7, 


0.6 


6. 8 


27.8 


Finland 


19.0 


11.2 


9. 1 


60.6 


13.8 


2.9 


0.9 


37.2 


Ireland 


18.6 


4.7 


17.0 


62.2 


9.0 


0.8 


9.0 


43. 5 


Italy 


-8.5 


10.5 


12.2 


56.3 










Netherlands 


15. 3 


20.2 


7.5 


55.2 


8.2 


7.9 


; 5.2 


31.2 


New Zealand 


14 '. 4 


14.7 


12.1 


60.0 


8.0 


0.2 


3. 3 


33.7 


Sweden 










7.5 


15.0 


6.0 


43.2 


U.S.A. 


20.4 


7.0 


2.8 


61.8 


18.3 


9.0 


4. 1 


56.7 ' 


Mean 


16. 3 


10.7 


11.2 . 


59.0 


9.9 


5.2 


5.0 


39.0 


Highest 


20.4 


2 b ;2 


17.7 


62.2 


18.3 


15.0 


9.0 


56.7 


Lowest 


8.5 


4.7 


2.8 


55.2 


• 4.7 


0.2 


0.9 


27.8 
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Appendixes 



Appendix VII ' " 

^1 



Contribution by blocks to total variance from between-student regression 
analyses - French Reading . 



Increment , Population II ' Population IV 



Country . 


Block 
1 ' 


Block 
2 


Block 
3 


Block 
4 


Total 


Block 
1 


Block - 
.2 


Block 
3 


Block 
4 


Total 


Chile . 












1.4 


3.9 


1.9 


7.2 


27,J. 


England 


12.0 


22. 1 


10.8 


3.6 


58,5. 


2.6 


4.9 


7.1 


6.0 


44.0 


Netherlands 


3.7 


12. 3 


14.0 


7.4 


42.4 


0.8 


0.4 


4.9 


5.0 


26.8 


New Zealand 


4.5 


1.5 


19.2 


5,5 


43.8 


2.0 


0. 0 


2.5 


14.3 


37.3 


-Romania 


10.6 


2.8 


0.0 


14.a 


33.0 


15.8 


8.6 


3.8 




47.4 


Scotland 


14.6 


12.3 


13.7 


9.8 


64.8 


2.7 


0.4; . 


. 5.1 


ief..6 


p2.8 


Sweden 












2.5 


0.0 


29.0 


22.5 


61.0 


U.S.A. 


2.6 


0.8 


4.1 


7.9 


?7.8 


3.3 


2.4 


7.1 


13.5 


47. 1 


Mean. 


8.0 


8.6 


8.6 


8. 2 


45. 1 


4.0 


2.6 


7.7 . 


11.7 


43.0 


Highest 


14.6 


22. 1 


14.0 . 


14.8 


64.8 


15.8 


8. 6 


29.0 


22.5 


61.0 


Lowest 


2.6 


0.8 


0.0 


3. 6 


27.8 


0.8 


0.0 


2.5 


5.0 


26.8 
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Appendix VIII 



Contribution by blocks to total variance from between-student regression 
analyses - English Readiilg 



Increment 



Population II 



Population IV 



Country ^"^n^ 


Block 
1 


31odk 
2 


Block 
3 


Block 
4 


Total 


Block 
1 


Block 
2 


Block 
3 


Block 
4 


total 


Belgium (French) 


14.2 


21.0 


6.5 


9.8 


57.0 


6.0 


3.4 


10.7 


15^6 


41.9 


' Chile 












8.2 


8.5 


4.7 


15.2. 


42.0 


Fed. Rep, of Germany 


15.1 


30.7 


14.0 


13.0 


78.5 


1.0 


0.5 


7.5 


8.9 


34.0 


Finland 


25.7 


40.4 


5.8 


2.2 


78.2 


4.3 


0.2 


19.3 


2.3 


43. 1 


Hunjgary 












2.9 


1^.7/ 


2. 1 . 


17.5 


41.6 


Israel . " 


22.4 


4. 1 


5.5 


4.4 


53.5 


9.7 


12.1 


7.7 


5.2 


42.3 


Italy:' 


1.2 


7.5 


5.4 


13.2 


36. I 


16.4 


30. 9 


5.8 


15;2 


75.0 


Netherlands. 


10:6 


51.7 


8.4 


2. 1 


77.5' 


0.6 


1. 2 


2.8 


5.8 


22.4 


Sweden 


12.5 


0.0 . 


'8.6 


4.2 


54,4 


3. 1 


19.5 


1.5 


2.3, 


47.0 


Thailand 


8.2 


1.4 


14.5 


9.9 


47.1 


. , L2.8 


5. 3 


5.0 


17. 1 


48.3 


Meah 

Highest 

Lowest 


13.7 
25.7 
1/2 


19.6 
51.7 
0.0 


8.6 
14.5 
5.4 


7.4 
13.2 
2. 1 


60.3 
.78.5 
36. 1 


6.5 
16.4-,, 
0.6 


9.4 

^3or&\ 

0.2 , 


6.7 
19.3 
1.5 


10.5 
17.5 
2. 3 


43.8 
75.0 
22.4 
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V 

ERIC 



OCCASIONAL PAPER No. 30: discusses the major results of a 
large-scale survey of educational achievement in twenty-two ' 
countries by the. International Association for the Evaluation of 
Educational Achievement (lEA). Testing .was undertaken in six 
different subject areas at two or three different levels in each 
school system. Particular attention is given to between -country 
differences in^achievement and factors accounting for differences 
between schools and between students within countries. 

T. NEVILLE POSTLETHWAITE Ph. D. (Stockholm) was Executive 
Director of the International Association for the Evaluation of 
Educational Achievement (lEA) from 19'J2-1972. He is currently 
an Associate Staff Member at the International Institute for 
Educational Planning in Paris, France. 
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